4 research outputs found
Learning Language-Conditioned Deformable Object Manipulation with Graph Dynamics
Multi-task learning of deformable object manipulation is a challenging
problem in robot manipulation. Most previous works address this problem in a
goal-conditioned way and adapt goal images to specify different tasks, which
limits the multi-task learning performance and can not generalize to new tasks.
Thus, we adapt language instruction to specify deformable object manipulation
tasks and propose a learning framework. We first design a unified
Transformer-based architecture to understand multi-modal data and output
picking and placing action. Besides, we have introduced the visible
connectivity graph to tackle nonlinear dynamics and complex configuration of
the deformable object. Both simulated and real experiments have demonstrated
that the proposed method is effective and can generalize to unseen instructions
and tasks. Compared with the state-of-the-art method, our method achieves
higher success rates (87.2% on average) and has a 75.6% shorter inference time.
We also demonstrate that our method performs well in real-world experiments.Comment: submitted to ICRA 202
Transparent Shape from a Single View Polarization Image
This paper presents a learning-based method for transparent surface
estimation from a single view polarization image. Existing shape from
polarization(SfP) methods have the difficulty in estimating transparent shape
since the inherent transmission interference heavily reduces the reliability of
physics-based prior. To address this challenge, we propose the concept of
physics-based prior, which is inspired by the characteristic that the
transmission component in the polarization image has more noise than
reflection. The confidence is used to determine the contribution of the
interfered physics-based prior. Then, we build a network(TransSfP) with
multi-branch architecture to avoid the destruction of relationships between
different hierarchical inputs. To train and test our method, we construct a
dataset for transparent shape from polarization with paired polarization images
and ground-truth normal maps. Extensive experiments and comparisons demonstrate
the superior accuracy of our method. Our codes and data are provided in the
supplements